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Audit  of  Major  Defense  Acquisition  Programs  Cycle  Time 


Executive  Summary 


Introduction.  The  Government  Performance  and  Results  Act  (GPRA)  of  1993,  Public 
Law  103-62,  initiated  program  performance  reform  with  a  series  of  pilot  projects  in 
setting  program  goals,  measuring  program  performance  against  those  goals,  pd 
reporting  publicly  on  progress  achieved.  This  report  is  one  in  a  series  resulting  from 
our  audits  of  GPRA  goals,  and  discusses  the  DoD  GPRA  Performance  Measure  2.4.2 
for  FY  2000,  Major  Defense  Acquisition  Program  Cycle  Time  (MDAP).  DoD 
established  the  objective  of  delivering  new  MDAPs  to  the  field  in  25  percent  less  time 
than  programs  initiated  prior  to  1992.  MDAPs  must  either  be  designated  by  the  Under 
Secret^  of  Defense  for  Acquisition,  and  Technology  USD  (A&T)  as  a  major  defense 
acquisition  program,  or  MDAPs  can  be  determined  by  USD  (A&T)  as  a  program  that 
requires  a  total  expenditure  of  $365  million  in  FY  2000  constant  dollars  for  research, 
development,  test,  and  evaluation  or  more  than  $2,190  billion  in  FY  2000  constant 
dollars  for  procurement.  The  cycle  time  goal  is  measured  by  the  average  monthly 
elapsed  time,  from  the  start  of  a  program  to  initial  operational  capability,  for  all 
MDAPs  in  development  during  a  specific  calendar  year.  DoD  reported  that  it  met  its 
FY  20(X)  goal  and  reduced  the  average  cycle  times  by  25  percent,  that  is,  to  less  than 
99  months,  from  the  historical  average  of  132  months. 

Objectives.  The  overall  objective  was  to  evaluate  the  MDAP  cycle  time  for  tracking 
performance  under  GPRA  as  indicated  in  the  FY  2000  Annual  Report  of  the  Secretary 
of  Defense.  Specifically,  we  assessed  whether  the  process  and  factors,  used  to  establish 
the  metric  goal  of  MDAP  cycle  time,  were  valid.  See  Appendix  A  for  a  discussion  of 
the  audit  scope,  methodology,  and  prior  coverage. 

Results.  The  database  used  to  calculate  MDAP  acquisition  cycle  time  for  inclusion  in 
the  FY  2000  Annual  Report  of  the  Secretary  of  Defense  to  the  President  and  the 
Congress,  was  not  accurate  or  complete.  Of  the  48  MDAPs  reviewed,  data  for 
28  programs  was  incorrect.  We  also  identified  three  programs  that  were  not  included 
in  &e  database.  As  a  result  of  our  findings,  USD  (AT&L)  has  contracted  for  the 
complete  verification  and  reconciliation  of  any  omissions  and  inconsistencies  in  the 
database.  As  of  December  2001,  USD  (AT&L)  estimated  that  it  will  complete  the 
verification  and  reconciliation  of  the  database  by  February  2002. 


Summary  of  Recommendation.  We  recommend  that  the  Under  Secretary  of  Defense 
for  Acquisition,  Technology,  and  Logistics  establish  a  quality  control  process  to 
periodically  review  the  MDAP  cycle  time  database  for  accurate  and  complete 
information. 

Management  Comments.  We  provided  a  draft  of  this  report  to  The  Under  Secretary 
of  Defense  for  Acquisition,  Technology,  and  Logistics  for  comments.  Comments  were 
not  received.  Therefore,  we  request  die  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  provide  comments  by  January  28,  2002. 
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SO  VO  l> 


Background 


The  Government  Performance  and  Results  Act  of  1993  (Public  Law  103-62). 

The  Government  Performance  and  Results  Act  (GPRA)  was  established  to 
improve  Government-wide  program  effectiveness,  accountability,  and, 
ultimately,  public  confidence  by  requiring  agencies  to  identify  measurable 
annual  performance  goals.  In  1998,  the  cycle  time  for  the  Major  Defense 
Acquisition  Program  (MDAP)  was  approved  as  a  measurement  for  tracking 
performance  under  GPRA,  Goal  2.4,  Improve  Acquisition.  The  milestone 
schedule  information  from  the  December  1996  Selected  Acquisition  Report 
(SAR)  established  the  historic  baseline  of  132  months.  The  programs  were 
separated  into  two  groups:  58  programs  that  started  before  1992  and 
48  programs  that  started  after  1992,  the  year  that  DoD  acquisition  reform 
began. 

Acquisition  cycle  time  for  an  individual  program  is  defined  as  the  period  from 
formal  initiation  of  an  acquisition  program  milestone  to  the  initial  operating 
capability.  In  1998,  DoD  established  the  goal  to  deliver  new  MDAPs  to  the 
field  in  25  percent  less  time-less  than  99  months— than  programs  initiated  before 
1992.  The  key  measure  for  that  goal  is  the  average  time  elapsed  from  program 
start  to  initial  operational  capability  for  all  MDAPs  in  development.  DoD 
computes  the  monthly  cycle  time  for  each  MDAP  program  and  combines 
individual  cycle  times  to  calculate  the  average  cycle  time  for  all  programs.  DoD 
reported  that  it  met  its  FY  2000  goal  of  reducing  the  average  cycle  time  for  all 
MDAPs  by  25  percent. 

Objectives 


The  overall  objective  was  to  evaluate  the  MDAP  cycle  time  for  tracking 
performance  under  GPRA  as  indicated  in  the  FY  2000  Annual  Report  of  the 
Secretary  of  Defense.  Specifically,  we  assessed  whether  the  process  and 
factors,  used  to  establish  the  metric  goal  of  MDAP  cycle  time,  were  valid.  See 
Appendix  A  for  a  discussion  of  the  audit  scope,  methodology,  and  prior  audit 
coverage. 
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Major  Defense  Acquisition  Programs 
Cycle  Time 

The  database  used  to  calculate  MDAP  acquisition  cycle  time  for 
inclusion  in  the  annual  report  to  the  President  and  the  Congress,  was  not 
complete  or  accurate.  We  identified  the  following  inconsistencies  that 
occurred  in  28  of  48  programs  (some  programs  had  multiple 
inaccuracies): 

•  20  programs  had  milestone  dates  that  differed  from  SAR  start 
dates, 

•  6  programs  had  different  low-rate  initial  production  dates, 

•  5  programs  had  different  initial  operational  capability  dates,  and 

•  12  programs  had  different  cycle  time  calculations. 

In  addition,  three  programs  were  not  included  in  the  database. 

Complete  and  accurate  information  was  lacking  because  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
(USD  [AT&L])  did  not  have  a  process  in  place  to  verify  information 
recorded  in  the  database  or  to  reconcile  any  discrepancies  between  the 
database  and  the  SARs.  As  a  result,  MDAP  cycle  time  information 
used  in  the  Annual  Report  of  the  Secretary  of  Defense  for  FY  2000  and 
the  Annual  Report  for  FY  2001  may  not  accurately  reflect  the  progress 
of  achieving  the  MDAP  cycle  time  goal. 

Acquisition  Performance  Goal  2.4 


The  typical  acquisition  effort  of  the  1960s  required  7  years  for  completion.  A 
review  of  MDAPs,  using  the  1996  SARs,  found  that  major  systems  required 
11  years  (132  months)  to  progress  from  program  start  to  initial  operational 
capability.  In  1998,  the  MDAP  cycle  time  was  approved  as  a  measurement  for 
tracking  performance  under  GPRA.  DoD  established  the  goal  of  delivering  new 
MDAPs  to  the  field  in  25  percent  less  time  (99  months)  from  the  historic 
average  (132  months).  USD  (AT&L)  is  responsible  for  monitoring  the  cost, 
schedule,  and  technical  status  of  major  weapon  systems.  In  1998,  USD  (AT&L) 
established  a  database  to  record  current  and  historic  schedule  information  for 
each  MDAP.  The  database  contained  information  used  to  calculate  the  average 
cycle  time  for  all  the  MDAPs  using  program  start  dates  and  initial  operational 
capability  dates.  Based  on  those  calculations,  USD  (AT&L)  computed  an 
average  cycle  time  of  96.9  months  for  48  MDAPs,  which  were  started  after 
1992,  and  were  reported  in  the  FY  2000  Annual  Report.  The  96.9  monthly 
average  cycle  time  showed  that  DoD  exceeded  its  established  goal  by  more  than 
2  months. 


2 


Database  Analysis 


We  could  not  verify  whether  DoD  met  GPRA  MDAP  cycle  time  goals  because 
the  data  used  to  calculate  MDAP  cycle  times  were  not  accurate  for  all  programs. 
We  compared  USD  (AT&L)  cycle  time  database  against  the  December  31, 

1999,  and  June  30,  2000  SARs.  The  database  contained  numerous  instances  of 
missing  or  inaccurate  information.  Appendix  B  shows  three  MDAPs  omitted 
from  the  database:  C-130J  Hercules,  Maneuver  Control  System,  and  Secure 
Mobile  Anti-Jam  Reliable  Tactical  Terminal.  Each  program  was  started  during 
or  after  1992,  had  SARs  prepared,  and  should  have  been  included  in  the 
database. 

Appendix  C  lists  eight  programs  in  the  database  that  used  “To  Be  Determined” 
as  an  initial  operational  capability  date.  However,  cycle  times  were  calculated 
for  four  of  the  eight  programs:  Airborne  Laser,  Evolved  Expendable  Launch 
Vehicle,  MILSTAR,  and  Space-Based  Infrared  System  Program  (High).  We 
were  unable  to  determine  cycle  times  for  those  programs  with  no  initial 
operational  capability  date. 

Appendix  D  shows  20  programs  with  milestone  dates  that  differed  from  SAR 
start  dates.  For  example,  die  database  listed  the  Cooperative  Engagement 
Capability  Program  with  no  date  for  Milestone  I,  yet  the  SAR  showed 
May  1995  for  Milestone  1.  Also,  the  ATACMS/BAT  Program  showed  a 
Milestone  II  date  of  May  1999,  although  the  SAR  showed  a  Milestone  II  date  of 
November  2000— a  difference  of  18  months.  Appendix  D  also  shows  that  six 
programs  had  different  low-rate  initial  production  dates  between  the  database 
and  the  SARs  and  five  programs  with  initial  operational  capability  dates  that 
differed  from  the  SARs.  For  example,  the  Force  XXI  Batde  Command  Brigade 
and  Below  Program  showed  an  initial  operational  capability  date  of  April  2002 
although  the  SAR  showed  November  2001 -a  difference  of  5  months. 
Additionally,  Appendix  D  shows  12  programs  each  with  a  different  cycle  time 
calculation  between  the  database  and  the  SARs,  such  as  the  Advanced 
Amphibious  Assault  Vehicle  Program. 

Cycle  Time  Database 

USD  AT&L)  established  the  database  to  record  DoD  schedule  information  for 
all  MDAPs  and  to  determine  cycle  time  for  the  programs.  Information  was 
collected  manually  from  SARs  and  transferred  to  the  database  where  it  became 
the  basis  for  determining  die  MDAP  average  cycle  time,  which  was  reported  in 
the  2000  Annual  Report  and  the  draft  of  the  2001  Annual  Report.  However, 
USD  (AT&L)  did  not  have  a  process  to  review  and  verify  the  completeness  and 
accuracy  of  the  database,  which  resulted  in  omissions  and  inaccuracies  in  the 
data.  An  independent  review  of  the  information  should  be  performed  before  it 
becomes  part  of  another  database  or  report. 

Based  on  information  taken  from  the  USD  (AT&L)  database,  DoD  reported  that 
it  had  met  its  FY  2000  goal  of  reducing  average  cycle  time  by  25  percent.  This 
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information  was  reported  to  the  President  and  the  Congress  in  the  2001  Annual 
Report,  Appendix  I,  Government  Performance  and  Results  Act,  DoD  FY  2000 
Performance  Report,  March  2001.  However,  because  programs  were  omitted 
and  other  discrepancies,  the  matrixes  showing  that  the  goal  was  reached  may  not 
accurately  reflect  the  progress  made  toward  achieving  Ae  cycle  time  goal.  The 
DoD  FY  2002  Aimual  Performance  Plan,  a  part  of  Appendix  I,  will  be 
published  separately  from  the  DoD  FY  2000  Performance  Report. 

Actions  Undertaken  by  the  Office  of  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  and  Logistics 


The  USD  (AT&L)  agreed  that  data  were  omitted  from  the  database  and  that 
other  data  were  inconsistent  with  information  in  the  SARs.  As  a  result,  a 
statement  of  work  was  issued  for  a  contractor  to  examine  and  upgrade  the 
database.  The  statement  of  work  addressed  the  following  tasks: 

•  Verify  and  reconcile  Inspector  General,  DoD,  discrepancies  in  the 
database  for  the  DoD  programs  currently  in  development,  and 
document  rules  for  determining  cycle  time. 

•  Verify  the  accuracy  of  schedule  data  for  the  remaining  programs  in 
the  database.  (There  are  about  200  completed  programs  dating  back 
to  1969.) 

•  Design  a  data  table  within  the  Consolidated  Acquisition  Reporting 
System  and  include  data  that  will  highlight  any  change  in  cost  and 
schedule. 

As  of  December  2001,  USD  (AT&L)  estimated  that  it  will  complete  the 
verification  and  reconciliation  of  the  database  by  February  2002. 

Sununary 


We  could  not  verify  whether  DoD  met  the  GPRA  metric  MDAP  cycle  time  goal 
because  the  database  used  to  calculate  the  average  cycle  time  omitted  programs 
and  contained  discrepancies.  As  a  result,  the  average  cycle  time  goal  stated  in 
the  FY  2000  Annual  Report  of  the  Secretary  of  Defense  may  not  be  accurate.  In 
addition,  cycle  times,  which  were  calculated  for  the  FY  2001  Annual  Report, 
using  information  from  the  same  database,  may  not  be  accurate.  Because  USD 
(AT&L)  representatives  took  action  to  reconcile  the  MDAP  database,  we  did  not 
make  a  recommendation  on  that  issue. 
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Recommendations 


We  recommend  the  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics  establish  a  quality  control  process  to  periodically  review  and 
verify  the  completeness  and  accuracy  of  the  MDAP  cycle  time  database. 

Management  Comments 


The  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  did 
not  provide  comments  to  the  draft  of  this  report,  dated  June  29,  2001. 
Accordingly,  we  request  that  the  Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics  provide  comments  to  the  final. 


5 


Appendix  A.  Audit  Process 

Scope 


We  evaluated  DoD  reporting  against  the  MDAP  cycle  time  performance  goal  of 
GPRA.  Specifically,  we  reviewed  Subordinate  Performance  Goal  2.4,  Improve 
Acquisition,  and  Performance  Measure  2.4.2,  MDAP. 

Cycle  Time.  Management  control  objectives  for  acquisition  program  cost, 
schedule,  and  performance  parameters  are  embodied  in  acquisition  program 
baselines. 

General  Accounting  Office  High-Risk  Area.  The  General  Accounting  Office 
has  identified  several  high-risk  areas  in  the  DoD.  This  report  provides  coverage 
of  the  Defense  Weapons  System  Acquisition  high-risk  area. 

Methodology 


Specifically,  we  reviewed  MDAPs  having  start  dates  after  1992.  We  obtained  a 
copy  of  the  MDAP  cycle  time  database  identifying  48  programs  and  their 
respective  cycle  times  from  USD  (AT&L).  We  reviewed  the  milestone 
schedules  for  the  51  MDAPs  listed  in  the  annual  SARs  to  determine  average 
cycle  times  for  each  of  the  programs.  We  recorded  each  of  the  MDAPs 
Milestone  I,  II,  and  III  dates,  low-rate  initial  production  dates,  and  initial 
operational  capability  dates  as  reported  in  the  SARs.  We  calculated  the  length 
of  time  elapsed  between  Milestone  I  or  Milestone  II  (some  MDAPs  use 
Milestone  I  as  a  start  date  and  others  use  Milestone  II  as  a  start  date)  and  initial 
operational  capability  dates  for  each  of  the  MDAPs  listed  in  the  SARs.  Also, 
we  compared  the  MDAP  cycle  time  that  we  determined  with  the  results 
calculated  in  the  USD  (AT&L)  cycle  time  database.  We  reconciled  the  MDAPs 
in  the  USD  (AT&L)  database  with  the  programs  in  the  SARs. 

Audit  Type,  Dates,  and  Standards.  We  performed  this  program  audit  from 
January  2001  through  June  2001  in  accord^ce  with  generally  accepted 
government  auditing  standards. 

Use  of  Computer-Processed  Data.  We  did  not  use  computer-processed  data  to 
perform  this  audit. 

Use  of  Technical  Assistance.  We  did  not  rely  on  technical  assistance  to 
perform  this  audit. 

Contacts  During  the  Audit.  We  visited  or  contacted  individuals  and 
organizations  within  DoD  and  contractor  locations.  Further  details  are  available 
upon  request. 
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Management  Control  Program  Review.  We  did  not  review  the  management 
control  program  for  GPRA  Performance  Measure  2.4.2  MDAP  Cycle  Time. 
We  limited  our  review  to  the  controls  over  the  reliability  of  data  taken  from  the 
Selected  Acquisition  Reports  and  recorded  in  the  USD  (AT&L)  MDAP  cycle 
time  database.  We  found  that  these  controls  were  not  adequate. 

We  identified  a  material  control  weakness  as  defined  by  DoD 
Instruction  5010.40.  Controls  were  not  adequate  to  ensure  that  data  taken  from 
the  Selected  Acquisition  Reports  and  recorded  in  the  USD  (AT&L)  MDAP 
cycle  time  database  were  correct  and  complete.  Specifically,  a  quality  control 
process  was  not  established  to  periodically  review  the  MDAP  cycle  time 
database  for  accurate  and  complete  information.  The  recommendation,  if 
implemented,  will  ensure  adherence  to  regulatory  requirements. 


Prior  Coverage 

General  Accounting  Office 

The  General  Accounting  Office  has  conducted  multiple  reviews  related  to 
GPRA.  Unrestricted  General  Accounting  Office  reports  can  be  accessed  over  the 
Internet  at  http://www.gao.gov. 

Inspector  General,  DoD 

Inspector  General,  DoD,  Report  No.  D-2001-128,  “Government  Performance 
and  Results  Act  Goals:  Surge  Sealift  and  Forces  Supported  by  Land-  and  Sea- 
Based  Pre-Positioning,”  May  23,  2001 

Inspector  General,  DoD,  Report  No.  D-2001-080,  “Government  Performance 
and  Results  Act  Goals:  Disposal  of  Excess  Real  ftoperty,”  March  15,  2001 

Inspector  General,  DoD,  Report  No.  D-2001-045,  “Government  Performance 
and  Results  Act  Goals:  Tank  Miles,”  February  7,  2001 

Inspector  General,  DoD,  Report  No.  D-2001-033,  “Government  Performance 
and  Results  Act:  Unfunded  Deport  Maintenance  Requirements,”  January  12, 
2001 

Inspector  General,  DoD,  Report  No.  D-2001-021,  “Government  Performance 
and  Results  Act  Reporting  on  Defense  Working  Capital  Funds  Net  Operating 
Results,”  January  10,  2001 

Inspector  General,  DoD,  Report  No.  D-2000-136,  “Reporting  of  Performance 
Measures  in  the  DoD  Agency- Wide  Financial  Statements,”  May  31,  2000 
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Appendix  B.  Programs  Omitted  From 

Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and 
Logistics  Database 


Program  Name 

Date 

Service 

Cycle  Time^ 
(in  months) 

C130J  Hercules 

1996 

Air  Force 

33 

Maneuver  Control  System 

1992 

Army 

64 

Secure  Mobile  Anti-Jam  Reliable 
Tactical-Terminal 

1992 

Army 

107 

’  Program  start  (MS  I,  MS  II  or  MS  HI)  to  program  initial  operational  capability  date. 


8 


Appendix  C.  Programs  with  To  Be  Determined 

as  Initial  Operational  Capability 
Date 

Program  Name 

Date 

Service 

Cycle  Time^ 
(in  months) 

Airborne  Laser 

1996 

Air  Force 

130^ 

Evolved  Expendable  Launch  Vehicle 

1996 

Air  Force 

71^ 

MILSTAR 

1992 

Air  Force 

120' 

Space-Based  Infrared  System  (High) 

1996 

Air  Force 

113' 

Interim  Armored  Vehicle 

2000 

Army 

*3 

Navy  Theater  Wide  Ballistic  Missile 

Defense 

1999 

DoD 

♦3 

Space-Based  Infrared  System  (Low) 

1999 

Air  Force 

*3 

Joint  Strike  Fighter 

1996 

DoD 

*3 

^  Program  start  (MS  I,  MS  II  or  MS  III)  to  program  initial  operational  capability  date. 

^  Programs  had  computed  cycle  time  despite  having  "To  Be  Determined"  as  an  initial  operational 
capability  date. 

^  Programs  did  not  compute  cycle  time  because  "To  Be  Determined"  was  an  initial  operational  capability 
date. 
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Appendix  D.  Summary  of  Differences  Between 

the  Under  Secretary  of  Defense  for 
Acquisition^  Technology,  and 
Logistics  Database  and  the  Selected 
Acquisition  Report 
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PtOTpr^m  Date  Service  Milestone  I  Milestone  H  Milestone  HI  .  UMP  IOC  Cycle  Time 

Cooperative  Engagement  Capability  1995  Navy  X 
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Tactical  Tomahawk  1998 


Appendix  E.  Report  Distribution 


Office  of  the  Secretary  of  Defense 

Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
Under  Secretary  of  Defense  (Comptroller) 

Deputy  Chief  Financial  Officer 
Deputy  Comptroller  (Program/Budget) 

Deputy  Under  Secretary  of  Defense  (Acquisition  Initiatives) 


Department  of  the  Army 

Auditor  General,  Department  of  the  Army 


Department  of  the  Navy 

Naval  Inspector  General 

Auditor  General,  Department  of  the  Navy 


Department  of  the  Air  Force 

Assistant  Secretary  of  the  Air  Force  (Financial  Management  and  Comptroller) 
Auditor  General,  Department  of  the  Air  Force 


Non-Defense  Federal  Organization 

Office  of  Management  and  Budget 


Congressional  Committee  and  Subcommittees,  Chairman  and 
Ranking  Minority  Member 

Senate  Committee  on  Appropriations 

Senate  Subcommittee  on  Defense,  Commerce  on  Appropriations 

Senate  Committee  on  Armed  Services 

Senate  Committee  on  Governmental  Affairs 

House  Subcommittee  on  Defense,  Committee  on  Appropriations 

House  Committee  on  Armed  Services 

House  Committee  on  Governmental  Reform 

House  Subcommittee  on  Government  Efficiency,  Financial  Management,  and 
Intergovernmental  Relations,  Committee  on  Governmental  Reform 
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Congressional  Conunittees  and  Subconunittees,  Chairman  and 
Ranking  Minority  Member  (Cont'd) 

House  Subcommittee  on  National  Security,  Veterans  Affairs,  and  International 
Relations,  Committee  on  Government  Reform 
House  Subcommittee  on  Technology  and  Procurement  Policy,  Committee  on 
Government  Reform 
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Audit  Team  Members 


The  Acquisition  Management  Directorate,  Office  of  the  Assistant  Inspector 
General  for  Auditing,  DoD,  prepared  this  report.  Personnel  of  the  Office  of  the 
Inspector  General,  DoD,  who  contributed  to  the  report  are  listed  below. 

Mary  L.  Ugone 
John  E.  Meling 
Michael  E.  Simpson 
George  A.  Leighton 
Ronald  L.  Nickens 
Timothy  J.Miller 
Trisha  L.  Staley 
Ann  A.  Ferrante 
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